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Parametric Cost Models 



Parametric cost models have several uses: 

• high level mission concept design studies, 

• identify major architectural cost drivers, 

• allow high-level design trades, 

• enable cost-benefit analysis for technology development 
investment, and 

• provide a basis for estimating total project cost. 


In the past 1 2 months 

Added JWST cost information for 2003, 2006, 2008 and 2009. 

Published two peer reviewed cost model papers: 

Stahl, H. Philip, Kyle Stephens, Todd Henrichs, Christian Smart, and 
Frank A. Prince, “Single Variable Parametric Cost Models for Space 
Telescopes”, Optical Engineering VoL49, No. 06, 2010 



Stahl, H. Philip, “Survey of Cost Models for Space Telescopes”, Optical 
Engineering , Vol.49, No. 05, 2010 


Now working on developing multi-variable cost models. 


Objectives for Today 


I 

i 

Review Data Collection Methodology 
Define Statistical Analysis Methodology 
Summarize Single Variable Results 
Test Historical Models 

Introduce Preliminary Multi-Variable Models 


Methodology 



Data on 59 different variables 
was acquired for 30 NASA, 
ESA, & commercial space 
telescopes using: 

• NAFCOM (NASA/ Air Force 
Cost Model) database, 

• RSIC (Redstone Scientific 
Information Center), 

• REDSTAR (Resource Data 
Storage and Retrieval System), 

• project websites, and interviews. 


Table 1: Cost Model Missions Database 

X-Ray Telescopes 

Infrared Telescopes 

Chandra (AXAF) 

CALIPSO 

Einstein (HEAO-2) 

Herschel 


ICES at 

UV/Optical Telescopes 

IRAS 

EUVE 

ISO 

FUSE 

JWST 

GALEX 

SOFIA 

HiRISE 

Spitzer (SIRTF) 

HST 

TRACE 

HUT 

WIRE 

IUE 

WISE 

Kepler 


Copernicus (OAO-3) 

Microwave Telescopes 

SOHO/EIT 

WMAP 

UIT 


WUPPE 

Radio Wave Antenna 


TDRS-1 


TDRS-7 


Missions 



Of the 30 mission, we initially 
studied 21 ‘normal- 
incidence’ UVOIR and 
Infrared telescopes. 

Of these, 

17 are ‘Free Flying’ and 
4 are ‘Attached’ 

To study wavelength diversity, 
we added microwave, radio 
wave and grazing incidence 
X-Ray/EUV. 
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Cost Variables 



Total Cost is Phase A through D, it does not include: 

• Phase E (post-launch) costs 

• Launch related costs 

• Civil servant costs (NASA employees) 

• So our Total Cost is contract cost to make the system. 
OTA Cost includes only: 

• Primary mirror 

• Secondary (and tertiary if appropriate) mirror(s) 

• Related support structure 

Total Mass and OTA Mass match the cost definitions 


Technical Variables 



Aperture Diameter 
Mass (OTA and Total) 

PM Focal Length 
PMF/# 

Field of View 

Pointing Accuracy 

Spectral Range Minimum 

Wavelength of Diffraction Limit 

Operating Temperature 

Average Input Power 

Data Rate 

Design Life 

Orbit 


Programmatic Variables 



Launch Year 

Year of Development (or Start of Development) 

Development Period 

TRL (Technology Readiness Level) 



Completeness of Data for 19 Variables 


Table 2: Cost Model Variables Study 
and the completeness of data knowledge 

Parameters 

% of Data 

OTA Cost 

89% 

Total Phase A-D Cost w/o LV 

84% 

Aperture Diameter 

100% 

Avg. Input Power 

95% 

Total Mass 

89% 

OTA Mass 

89% 

Spectral Range 

100% 

Wavelength Diffraction Limit 

63% 

Primary Mirror Focal Length 

79% 

Design Life 

100% 

Data Rate 

74% 

Launch Date 

100% 

Year of Development 

95% 

Technology Readiness Level 

47% 

Operating Temperature 

95% 

Field of View 

79% 

Pointing Accuracy 

95% 

Orbit 

89% 

Development Period 

95% 

Average 

88% 


Model Creation 



Start with Correlation Matrix. 

Look for Variables which are Highly Correlated with Cost. 

The higher the correlation the greater the Cost Variation which is 
explained by a given Variable. 

Sign of correlation is important and must be consistent with Engineering 
Judgment. 

Important for Multi- Variable Models: 

We want Variables which Independently effect Cost. 

When Variables ‘cross-talk’ with each other it is called Multi-Collinearity. 
Thus, avoid Variables which are highly correlated with each other. 


Goodness of Correlation, Fits and Regressions 

‘Correlation’ between variables and ‘Goodness’ of single variable 
models is evaluated via Pearson’s r 2 standard percent error 
(SPE), and Student’s T-Test p- value. 

‘Goodness’ of multivariable fits are evaluated via Pearson’s 
Adjusted r 2 which accounts for number of data points and 
number of variables. 

Pearson’s r 2 coefficient describes the percentage of agreement 
between the fitted values and the actual data. 

The closer r 2 is to 1, the better the fit. 

SPE is a normalized standard deviation of the fit residual 
(difference between data and fit) to the fit. 

The closer SPE is to 0, the better the fit 



Significance 



The final issue is whether or not a correlation or fit is significant. 

p-value is the probability that the fit or correlation would occur if 

the variables are independent of each other. 

The closer p-value is to 0, the more significant the fit or correlation. 

The closer p-value is to 1, the less significant. 

If the p-value for a given variable is small, then removing it from the 
model would cause a large change to the model. 

If p-value is large, then removing the variable will have a negligible effect 

It is only possible to ‘test’ if the correlation between two 
variables is significant. 

It is not possible to ‘test’ if two variables are independent. 


Cross Correlation Matrix 
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Correlation Significance Details 


Parameter 

Total Cost 

OTA Cost 

OTA Areal Cost 

Cor 

P 

N 

Co IT 

P 

N 

Corr 

P 

N 

Diameter 

.68 

.007 

14 

.87 

0 

16 

-.71 

.005 

14 

Focal Length 

.82 

.002 

11 

.82 

.001 

12 

-.42 

.194 

11 

Pointing Accuracy 

-.53 

.061 

14 

-.64 

.011 

15 

.47 

.087 

14 

Total Mass 

.92 

0 

15 

.68 

.005 

15 

-0 

.997 

15 

OTA Mass 

.72 

.002 

15 

.82 

0 

15 

-.47 

.074 

15 

Spectral Min 

-.02 

.934 

16 

.07 

.804 

17 

-.23 

.383 

16 

Operating Temp 

-.04 

.884 

16 

0 

.975 

16 

-.07 

.802 

16 

Electrical Power 

.59 

.021 

15 

.14 

.611 

16 

-.05 

.862 

16 

Design Life 

.65 

.007 

16 

.46 

.064 

17 

-.20 

.454 

16 

TRL 

-.41 

.307 

8 

-.68 

.061 

8 

-.29 

.481 

8 

Development Period 

.78 

.001 

15 

.45 

.083 

15 

.14 

.830 

15 

Launch Year 

.11 

.675 

16 

-.16 

.533 

17 

-.34 

.204 

16 


Diameter appears to be the most significant cost driver. So, in addition to total 
cost and OTA cost we have examined OTA Areal Cost, i.e. OTA Cost per 
unit Area of Primary Mirror collecting aperture. Diameter is correlated 
with all three with a significance of greater than 99%. 



Correlation Significance Details 
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P 
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Primary Mirror Focal Length is also a significant correlation, but as we will 
discover later, it is multi-collinear with Diameter. The assumed explanation 
is that all space telescopes tend to have the same basic PM F/#. 



Correlation Significance Details 
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Pointing Accuracy has reasonably correlation with cost. And, as expected 
from engineering judgment, it has significant correlation (99% confidence 
level) with diameter and OTA mass. Interesting, as will be discussed later, 
pointing is not multi-collinear with either. 



Correlation Significance Details 
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As expected, Total Mass correlates most significantly with Total Cost while 
OTA Mass correlates most significantly with OTA Cost. 



Correlation Significance Details 
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16 


Unexpectedly, Minimum Spectral Range Value and Operating Temperature do 
not have a significant correlation with any Cost. However, as we will show 
later, Spectral Minimum does have a role in multi- variable cost models. 



Correlation Significance Details 
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As expected Electrical Power, Design Life and Development Period have 
significant correlations (99% confidence) with Total Cost. 



Correlation Significance Details 
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Also unexpected is that TRL and Launch Year do not have significant 
correlations. But, as we will discuss later, they both have roles in multi- 
variable cost models. One problem with TRL is there are only 8 data 
points. 


OTA Cost or Total Cost 

Engineering judgment says that OTA cost is most closely related 
to OTA engineering parameters. But, managers and mission 
planners are really more interested in total Phase A-D cost. 

For 14 missions free flying missions, 

OTA cost is -20% of Phase A-D total cost (R 2 = 96%) 

with a model residual standard deviation of approximately $300M. 




Total Cost vs OTA Cost/Total Cost 
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OTA Cost or Total Cost 



We have detailed WBS data for 7 of the 14 free flying missions. 
Mapping on common WBS indicates that OTA is -30% of Total, 


Typical Space Telescope Cost Breakdown 
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OTA Cost vs Aperture Diameter 

For free-flying space telescopes: 

OTA Cost - Aperture Diameter 1 - 28 (N = 16; r2 = 84%) without JWST 

OTA Cost - Aperture Diameter 1 - 2 (N = 17; r2 = 75%) with 2009 JWST 



Area Cost 



Total Cost is important, but Areal Cost might be more relevant. 

Areal Cost decreases with aperture size, therefore, larger 
telescopes provide a better ROI 

OTA Areal Cost ~ Aperture Diameter '°- 74 (N=i7;r 2 = 55%) with jwst 



Mass Models 



While aperture diameter is the single most important parameter 
driving science performance. 

Total system mass determines what vehicle can be used to launch. 

Significant engineering costs are expended to keep a given 
payload inside of its allocated mass budget. 

Such as light-weighting mirrors and structure. 


Space telescopes are designed to mass 


Mass Models 



Our data shows that 

Total Mass is ~ 3.3X OTA Mass (r 2 = 92%), and 
Total Cost is ~3.3X to 5X OTA Cost. 

3.3X comes from WBS analysis 
5X comes from regression analysis 


Mission Mass Ratio Cost Ratio 

JWST -2.6X -5.3X 

Hubble 4.6X 5.5X 

Chandra 6.2X 2.8X 


For Chandra, science instruments were massive and optics expensive 


Total Cost vs Total Mass 

Based on 15 free-flying OTAs 

Total Cost - Total Mass 112 (N = 15; r 2 = 86%) with jwst 
Total Cost - Total Mass 104 (N = u ; r 2 = 95%) without jwst 



OTA Cost vs OTA Mass 



Based on 15 free-flying OTAs 

OTA Cost - OTA Mass 0 69 (N = 14; r 2 = 84% ) without JWST 
OTA Cost - OTA Mass °- 72 (N = 15; r 2 = 92%) with JWST 


OTA Cost vs OTA Mass 
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It costs more to make a Lightweight Telescope 

For 1 5 free-flying and 4 attached missions 

(3 to Space Shuttle Orbiter and SOFIA to Boeing 747) 

‘Attached’ OTAs are ~10X more massive than ‘free-flying’ 
‘Attached’ OTAs cost -60% less than ‘free-flying’ 




Problem with Mass 



Mass may have a high correlation to Cost. 

And, Mass may be convenient to quantify. 

But, Mass is not an independent variable. 

Mass depends upon the size of the telescope. 

Bigger telescopes have more mass and Aperture drives size. 

And, bigger telescopes typically require bigger spacecraft. 

The correlation matrix says that Mass is highly correlated with: 
Aperture Diameter, Focal Length, F/# , Volume, Pointing and Power 

But in reality it is all Aperture, the others all depend on aperture. 


Single Variable Model Statistical Summary 



While Mass regression has the highest correlation (Pearson’s r 2 ), 
it also has the highest uncertainty (SPE). 


Table 4: Summary of Single Variable Cost Model Statistics 


OTA Cost 

OTA Areal Cost 

OTA Cost 

Total Cost 

Variable 

OTA Diameter 

OTA Diameter 

OTA Mass 

Total Mass 

includes JWST 

yes 

no 

yes 

no 

yes 

no 

yes 

no 

Exponent 

1.2 

1.28 

-0.74 

-0.72 

0.72 

0.69 

1.12 

1.04 

Coefficient 

98.5 

103.5 

122.0 

133.6 

1.03 

1.58 

0.16 

0.24 

S log$ 

0.62 

0.64 

0.62 

0.64 

0.70 

0.70 

0.53 

0.54 

Pearson's r 2 

75% 

84% 

55% 

52% 

92% 

84% 

86% 

95% 

SPE 

79% 

79% 

78% 

79% 

93% 

91% 

71% 

77% 

n 

17 

16 

17 

16 

15 

14 

15 

14 


Multivariable Model required to increase r 2 and decrease SPE. 


Testing Historical Models 

Of all the historical models, the Horak model is the easiest to test. 
Our database has parameters equal to the Horak database. 

Horak published statistical fit details. 

CER: Tl ■ 0,157 (MatlKDsnXApr) 0 705 (#Elem)° 473 (J.)' 0 - ,TB (K P )*°' 191 3( Yr-80) 

T Statistics: (1 74) (8.80) (255) (-2.04) (-2 61) (-2 31) 

Sluistics R ! = 97.0% 

s = 0212 (17 Data Points) 

For this effort, we will ignore the Material (glass vs metal) and 
Design (on vs off-axis) multiplier factors. 



Testing the Horak Model 

Horak model with p- value significance analysis: 

71 ~ Apr^Elem^A^K^^e^^ 

T Statistics: 8.80 2.55 -2.04 -2.61 -2.31 

p-values: 0.00 0.022 0.059 0.020 0.036 

Based on 17 data points, all variables in model are significant and 
the fit has a good R 2 = 97% . 

Testing Horak against our Data Base yields: 

OTA Cost ~ 

T Statistics: 9.34 -1.03 -0.22 -0.38 -2.80 

p-values: 0.00 0.320 0.829 0.710 0.014 

Based on 16 data points, only Diameter and Launch Year are significant and 
the fit has a good R 2 =90.8% and r 2 ad j=86.2%. 



Testing the Horak Model 

Eliminating the insignificant variables yields: 

OTACost- Diam 

T Statistics: 10.61 -4.22 

p-values: 0.00 0.001 

Based on 17 data points, both Diameter and Launch Year are significant and 
the fit has a good R 2 =89.2% and r 2 ad j=87.6%. 



The explanation is in the databases. 

Horak’s database consisted mostly of DoD strategic systems most of which were 
laboratory experiments that were never deployed. Of the systems which were 
flown, most were airframe or missile systems. 


Our database consists entirely of NASA space telescope missions. 


Multi- Variable Models 



Starting with Single Variable Model for OTA Cost vs Diameter: 

OTA Cost - Aperture Diameter 1 - 28 (N = 16; r2 = 84%) without jwst 
OTA Cost - Aperture Diameter 1 - 2 (N = 17; r2 = 75%) with 2009 jwst 

Perform multi-variable regression to add a second variable. 

Select multi- variable model based on: 

Change in Significance of Diameter to Fit 
Significance of Variable #2 to Fit 
Increase in r 2 adj 
Decrease in SPE 
Multi-Collinearity 

Some variables may increase r 2 adj and/or decrease SPE, but they are not 
significant or their coefficients are not consistent with engineering 
judgment or they are multi-collinear. 


Multi- Variable Models 



There are two second variables with best meet all the critieria: 
Year of Development, and 
Launch Year 

Launch Year has the advantage that it is a definite date, but it also 
has the disadvantage that a launch can be delayed. And, while 
a launch delay tends to increase the total mission cost, it may 
or may not increase the OTA cost. 

Year of Development yields a slightly better regression, but its 
exact date is subject to definition. Is it the Start of Phase A or 
BorC? 

TRL has a significant result that yields an improved r 2 , but it does 
not reduce SPE. This is probably because of the relatively few 
data points. 


OTA Cost versus Diameter and V 2 
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T3 
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<D re 
^ > 

Diameter 
Second Variable 
Adjusted r2 
SPE 

PM F Len. 

PM F/N 

CD 

E 

o 

> 
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1- 
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FOV 

Pointing 

Accuracy 

OTA Mass 

OTA Areal 
Density 

Spectral Range 
minimum 

Wavelength 

Diffraction 

Limit 

Operating 

Temperature 

0.68 

0.27 

1.05 

0.00 

-0.02 

0.99 

1.16 

0.01 

1.14 

0.00 

0.76 

0.12 

1.45 

0.00 

1.22 

0.00 

1.19 

0.00 

1.21 

0.00 

0.35 

0.45 

0.26 

0.57 

0.35 

0.45 

-0.26 

0.18 

-0.05 

0.45 

0.35 

0.26 

0.35 

0.26 

-0.04 

0.63 

-0.10 

0.55 

-0.01 

0.96 

71% 

71% 

71% 

14% 

73% 

83% 

83% 

73% 

75% 

71% 

77% 

78% 

77% 

73% 

78% 

83% 

83% 

84% 

95% 

82% 

n 

13 

13 

13 

13 

16 

15 

15 

17 
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Avg. Input 
Power 

Data Rate 

Design Life 

Design Life 
(exp) 

Technology 
Readiness Level 

YoD (exp) 

Development 

Period 

Dev Per (exp) 

Launch Date 
(exp) 

Orbit 

1.41 

0.00 

1.40 

0.00 

1.21 

0.00 

1.13 

0.00 

1.31 

0.00 

1.27 

0.00 

1.19 

0.00 

1.20 

0.00 

1.34 

0.00 

1.23 

0.00 

-0.15 

0.23 

-0.08 

0.28 

-0.01 

0.98 

0.00 

0.51 

-0.09 

0.02 

-0.04 

0.00 

0.23 

0.60 

0.00 

0.73 

-0.04 

0.00 

0.02 

0.62 

70% 

91% 

71% 

84% 

97% 

95% 

71% 

71% 

93% 

66% 

58% 

59% 

83% 

81% 

83% 

39% 

77% 

78% 

39% 

85% 

n 

16 

12 

17 

17 
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16 

16 

17 
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No 

No 

No 

No 

No 
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Diameter and Year of Development 



OTACost = $332 M * Diameter 121 * e -° 038 *(^ 196 °) 


Pearson’s r 2 ad j=95% 

SPE=39% 

All coefficients are significant (p-values all <0.01). 

No evidence of non-constant variance or non -normality. 


Residual Plots for OTA Cost 


Normal Probability Plot 


Versus Fits 
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OTA Cost vs Diameter, YoD and V3 
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Third 

Variable 
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Density 

Spectral 
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SPE 

1.62 

0.01 

1.29 

0.00 

2.29 

0.07 

1.27 

0.00 

1.27 

0.00 

1.45 

0.00 

1.22 

0.00 

1.27 

0.00 

1.34 

0.00 

-0.04 

0.01 

-0.04 

0.01 

-0.04 

0.01 

-0.05 

0.01 

-0.04 

0.00 

-0.04 

0.00 

-0.04 

0.00 

-0.04 

0.00 

-0.05 

0.01 

-0.33 

0.34 

-0.33 

0.30 

-0.33 

0.34 

-0.01 

0.96 

0.00 

0.97 

-0.11 

0.63 

-0.11 

0.63 

0.01 

0.81 

-0.01 

0.90 

95% 

95% 

95% 

82% 

95% 

92% 

92% 

94% 

99% 

42% 

41% 

42% 

45% 

41% 

42% 

42% 

40% 

49% 

n 

12 

12 

12 

12 

16 

14 

14 

16 


Multicollinearity? 
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Operating 

Temperature 

Avg. Input 
Power 

Data Rate 

Design Life 

Design Life 
(exp) 

Technology 

Readiness 

Level 

Development 

Period 

Dev Per (exp) 

Orbit 

1.27 

0.00 

1.31 

0.00 

1.22 

0.00 

1.22 

0.00 

1.20 

0.00 

1.27 

0.00 

1.29 

0.00 

1.32 

0.00 

1.28 

0.00 

-0.04 

0.00 

-0.04 

0.01 

-0.03 

0.02 

-0.04 

0.00 

-0.04 

0.00 

-0.01 

0.29 

-0.04 

0.00 

-0.04 

0.00 

-0.04 

0.00 

0.00 

0.98 

-0.05 

0.59 

0.01 

0.82 

0.10 

0.45 

0.00 

0.38 

-0.71 

0.09 

-0.11 

0.72 

0.00 

0.59 

0.02 

0.42 

94% 

86% 

90% 

97% 

97% 

99% 

95% 

95% 

92% 

41% 

41% 

39% 

40% 

40% 

33% 

41% 

40% 

35% 

n 
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15 
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16 
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16 

15 

Multicollinearity? 

No 

No 

No 

No 

No 

No 

No 

No 

No 


Three-variable Models 



None of the three- variable models are better than the base model. 


While TRL looks promising, more data is needed. Also, adding 
TRL reduces the significance of YoD. 


But what if we add more high and low wavelength telescopes to 
gain some wavelength diversity? 

WMAP, 

TDRS-1, TDRS-7, 

EUVE, 

Chandra and Einstein 



OTA Cost vs Diameter, YoD and Spectral Minimum 
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Diam 

Diam*spct 

Diam*YoD 

all 3 

Diameter 

0.84 

0.00 

1.03 

0.00 

0.78 

0.00 

1.15 

0.00 

YoD (exp) 

- 

- 

- 

- 

-0.03 

0.12 

-0.03 

0.04 

Spot Min 

- 

- 

-0.13 

0.00 

- 

- 

-0.17 

0.00 

Adjusted r2 

43% 

69% 

18% 

92% 

SPE 

126% 

88% 

97% 

76% 

n 

23 

23 

20 

20 

Multicollinearity? 

N/A 

No 

No 

No 


Interestingly, adding wavelength diversity to the regression yields 
coefficients similar to the Horak model: 

71 ~ 


Conclusions 



From engineering & science perspective, Aperture Diameter is 
the best parameter for a space telescope cost model. 

But, the single variable model only predicts 84% of OTA Cost: 

OTA Cost ~ D 1 3 (N = 16; r2 = 84%; SPE=79%) without JWST 

OTA Cost ~ D 1 2 (N = 17; r2 = 75%; SPE=79%) with 2009 JWST 

Two Variable Models provide better estimates 

OTA Cost ~ D 1 3 e - |ul4(LYr - 1%1111 (N = 17, r 2 = 93%; SPE=39%) 

OTA Cost ~ D 1 3 e‘°- 04(YoD ' 196#)) (N = 16, r- = 95%; SPE=39%) 

A potential Three Variable Model is: 

OTA Cost ~ D 1 15 /A- 17 e - ##3(Yol) - 1%lll) (N = 20, r- = 92%; SPE = 76%) 

Finally, OTA mass is not a good CER 

OTA mass is multi-collinear with diameter, and 
more massive telescopes actually cost less to make. 


